Freeform drawing beautification

ABSTRACT

Freeform drawing beautification techniques are described. An input is received by a computing device describing a freeform path drawn by a user as part of a drawing, the freeform path not formed solely as a circular arc or a circle (e.g., a fixed distance from a point) and including one or more curved elements. The drawing is examined by the computing device to locate another curved element in the drawing. One or more suggestions are constructed to adjust the freeform path by the computing device based on the located curved element in the drawing. The constructed one or more suggestions are output to adjust the freeform path by the computing device.

BACKGROUND

Freeform drawings are one of the quickest ways for a user to createcontent to graphically represent an image, idea or principle, because auser is free to provide an input without constraint. Freeform drawingsare also referred to freehand drawing, sketching, and so on and aregenerally unconstrained inputs specified using freeform paths in amanner that mimics a user's ability to draw on a piece of paper using apen or pencil. In computing devices, a user may define the freeformpaths that is detected using touchscreen functionality of a displaydevice (e.g., as a gesture, through use of a stylus), use of a cursorcontrol device, and so on.

Although a user may quickly input these freeform paths to compose adrawing, the freeform paths are typically informal and thus not intendedas a finished work. Conventional techniques have been developed toincrease precision and look-and-feel of these freeform drawings, howeverthese conventional techniques typically require detailed knowledge onthe part of a user are involve a significant amount of time to perform.

SUMMARY

Freeform drawing beautification techniques are described. An input isreceived by a computing device describing a freeform path drawn by auser as part of a drawing, the freeform path not formed solely as acircular arc or a circle (e.g., a fixed distance from a point) andincluding one or more curved elements. The drawing is examined by thecomputing device to locate another curved element in the drawing. One ormore suggestions are constructed to adjust the freeform path by thecomputing device based on the located curved element in the drawing. Theconstructed one or more suggestions are output to adjust the freeformpath by the computing device.

In one or more implementations, an input is received by a computingdevice describing the freeform path drawn by a user as part of adrawing. The drawing is examined by the computing device to locate otherpaths that already exist in the drawing. A likelihood is evaluated thatthe freeform path conforms individually to respective ones of aplurality of geometric rules, one or more of which consider the otherpaths that already exist in the drawing such that the evaluatingperformed by at least one said geometric rule causes output of amodification of the freeform path and a score representing a likelihoodthat the modification is correct. One or more suggestions areconstructed to adjust the freeform path by the computing device based onthe score generated by the rules and the constructed suggestions areoutput to adjust the freeform path by the computing device.

In one or more implementations, a system includes a freeform drawingmodule implemented at least partially in hardware, the freeform drawingmodule configured to receive an input describing a freeform path drawnby a user as part of a drawing. The freeform path may include one ormore curved elements. The system also includes a beautificationassistant module implemented at least partially in hardware. Thebeautification assistant module is configured to construct one or moresuggestions to adjust the freeform path based on geometric principlessuch as straightness, circularity, parallelism, congruence,transformation, and curve offset, possible relative to paths previouslyincluded in the drawing.

A computing device includes one or more modules implemented at leastpartially in hardware. The one or more modules are configured to performoperations including receiving an input by the computing devicedescribing a freeform path drawn by a user as part of a drawing andconstructing one or more suggestions by the computing device to adjustthe freeform path based on curve similarity or as a curve offset inreference to at least one located curved elements previously included inthe drawing.

This Summary introduces a selection of concepts in a simplified formthat are further described below in the Detailed Description. As such,this Summary is not intended to identify essential features of theclaimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Theuse of the same reference numbers in different instances in thedescription and the figures may indicate similar or identical items.Entities represented in the figures may be indicative of one or moreentities and thus reference may be made interchangeably to single orplural forms of the entities in the discussion.

FIG. 1 is an illustration of an environment in an example implementationthat is operable to employ freeform drawing beautification techniquesdescribed herein.

FIG. 2 depicts an example of a beautification workflow of abeautification assistant module of FIG. 1.

FIG. 3 depicts an implementation showing examples of geometric rulesthat are supported by the beautification assistant module of FIG. 1.

FIG. 4 depicts an example implementation of successive rule evaluationand application performed by the beautification assistant module of FIG.1.

FIG. 5 depicts an example of graph pruning.

FIG. 6 depicts an example of reduction of memory and computationcomplexity using path sampling.

FIG. 7 depicts an example implementation in which a discrete Frechetdistance is shown.

FIG. 8 depicts an example implementation showing adjustment of alignmentof two tested paths.

FIG. 9 depicts an example implementation of offset path detection.

FIG. 10 depicts an example implementation showing a transform againfeature.

FIGS. 11, 12, 13, and 14 depict respective examples of techniquesperformed by a beautification assistant module of FIG. 1.

FIG. 15 is a flow diagram depicting a procedure in an exampleimplementation of controlling beautification of freeform drawings by acomputing device.

FIG. 16 is a flow diagram depicting another procedure in an exampleimplementation of controlling beautification of freeform drawings by acomputing device.

FIG. 17 illustrates an example system including various components of anexample device that can be implemented as any type of computing deviceas described and/or utilize with reference to FIGS. 1-16 to implementembodiments of the techniques described herein.

DETAILED DESCRIPTION Overview

Freeform drawings such as sketching are one of the simplest ways tovisualize ideas. One key advantage is that the user is not required tohave specialized knowledge of particular drawing software nor anyadvanced drawing skills. In practice, however, these skills may becomenecessary to improve the visual fidelity of the resulting drawing.Conventional techniques used to make the freeform input look moreprecise and polished, however, require the user to have experience withdrawing software. Even if the user has this experience, use ofconventional drawing software could take even longer to perform thanmaking the freeform input itself.

Freeform drawing beautification techniques are described that are usedto bridge the gap between imprecise sketches and complicated use ofconventional drawing software. These techniques convert an imprecisefreeform input to a precise output by exploiting rules that are able toascertain a likely intent of a user that provided the freeform input.The rules, for instance, are usable to construct suggestions that arestraightened or made into arcs, or are perpendicular, parallel, offset,translated, scaled, or rotated to other paths already drawn by a user,if present. Considerations may also be made for document zoom level,distance from existing art (e.g., closer art influences the suggestionsmore than distance art), age of the art (e.g., recently drawn artinfluences suggestions more than older art), and so forth.

For example, suggestions may be made by a computing device for a newlycreated path by comparing it to existing paths. The set of suggestionstakes into account parallelism, perpendicularity, coordinate snapping,path length, axis alignment, line reflection, circularity, curve offset,smooth curve tangents, and similarity to other paths.

To determine curve offset suggestions, a distance from a new path toexisting paths is determined and situations are found where the distanceis relatively constant. If the distance is relatively constant, a newpath is suggested that is evenly offset.

A variety of techniques may be used to determine curve similarity. Forexample, an examined curve may be scaled and aligned in a way that theendpoints match the endpoints of an existing path. A distance metric isused to scale a length of the existing path, thus giving a scaleindependent distance. If the relative distance is sufficiently small,the examined curve is replaced with a scaled, rotated, and/or translatedcopy of the existing curve.

Additionally, zoom level may be considered using screen space for alldistances, thereby making the suggestions scale independent. A varietyof other examples are also contemplated, further discussion of which iscontained in the following sections.

Techniques are also described to control adjustments made to thefreeform path based on geometric rules. For example, a directed graphmay be formed by a computing device by treating each of a plurality ofgeometric rules as an extensible set of self-contained geometric rules,each built as a black box and independent of other rules. Examples ofgeometric rules include endpoint snapping, end tangent alignment, lineparallelism, line perpendicularity, line length equality, arc centersnapping, curve offset, curve identify, curve rotation, or curvereflection. The geometric rules may be chained in the directed graph tosupport complex modifications of the input freeform path throughsuccessive application of the geometric rules to arrive at modificationsand scores for those modifications by “walking” the directed graph usingscores indicating a likelihood that a corresponding modification iscorrect to arrive at a final adjustment, further discussion of which isincluded in relation to FIGS. 3, 4, and 16.

In the following discussion, an example environment is first describedthat may employ the techniques described herein. Example procedures arethen described which may be performed in the example environment as wellas other environments. Consequently, performance of the exampleprocedures is not limited to the example environment and the exampleenvironment is not limited to performance of the example procedures.

Example Environment

FIG. 1 is an illustration of an environment 100 in an exampleimplementation that is operable to employ freeform drawingbeautification techniques described herein. The illustrated environment100 includes a computing device 102 having a display device 104, onwhich, a user interface may be displayed to support user interaction.

The computing device 102 may be configured in a variety of ways. Thecomputing device 102, for instance, may be configured as a desktopcomputer, a laptop computer, a mobile device (e.g., assuming a handheldconfiguration such as a tablet or mobile phone as illustrated), and soforth. Thus, the computing device 102 may range from full resourcedevices with substantial memory and processor resources (e.g., personalcomputers, game consoles) to a low-resource device with limited memoryand/or processing resources (e.g., mobile devices). Additionally,although a single computing device 102 is shown, the computing device102 may be representative of a plurality of different devices, such asmultiple servers utilized by a business to perform operations “over thecloud” as further described in relation to FIG. 17.

The computing device 102 is illustrated as including a freeform drawingmodule 106. The freeform drawing module 106 is representative offunctionality of the computing device 102 to accept and process userinputs to form freeform drawings for display by the display device 104.A user, for example, may provide inputs using a cursor control device,gesture, stylus, and so on to define a freeform drawing for output. Thefreeform drawing module 106 is also illustrated as including abeautification assistant module 108 that is representative offunctionality to automatically rectify a freeform drawing input by auser in a manner that maintains simplicity and speed of freeformsketching while still taking into account implicit geometric relations.

Examples of functionality of the beautification assistant module 108 areillustrated as a general curve assistant module 110 to supportrectification of freeform drawings having general curves 112 (e.g.,supports poly-curves composed of general cubic Bezier curves in additionto simple line segments and arcs), a curve offset module 114 to supportfreeform drawing of curve offsets 116, and an affine transformationmodule 118 that supports freeform drawings having affine transformations120, e.g., by applying a linear combination of translation, rotation,scaling, reflection, and/or shearing. Other examples are alsocontemplated as further described in the following.

FIG. 2 depicts an example 200 of a beautification workflow of thebeautification assistant module 108 of FIG. 1. Incremental beatificationis performed by the beautification assistant module 108 is shown assuccessive input lines 202-214 and adjusted lines 204′-214′ throughstages 1-7 to arrive at a beautification result 216, as contrasted withdrawn collection 218 of input lines 202-214.

For example, a user initially draws input line 202, which remains as isbecause there are no other lines that serve as a basis for adjustment.Input line 204 is then drawn, which is adjusted by the beautificationassistant module 108 to form adjusted line 204′ to form a mirror imageof input line 202 through a process referred to as reflection. Inputline 206 is then drawn by a user, which is then adjusted by thebeautification assistant module 108 to have a symmetric curve as shownfor adjusted line 206′ through arc fitting. Similarly, input line 208 isdrawn by a user and adjusted by the beautification assistant module 108to have a symmetric curve as shown for adjusted line 208′.

An input line 210 is then drawn to represent a plant leaf, which issufficiently different from other lines and as such as not adjusted.Curve identity, reflection, and scaling are then used to adjustsubsequent input lines 212, 214 to form adjusted lines 212′, 214′ thatare based at least in part on adjusted line 210′ and as such mimiccharacteristics of the plant leaf.

Thus, as the freeform drawing continues, more suitable geometricconstraints emerge and are applied by the beautification assistantmodule 108 as suggestions, such as curve identity (e.g., stages 2, 6,7), reflection (e.g., stages 2 and 6) and arc fitting (stages 3,4).Thus, as is readily apparent through comparison with the input 218 andfinal output freeform art at the result 216 the beautification assistantmodule 108 supports efficient freeform drawing with results that areready for sharing with others.

FIG. 3 depicts an implementation 300 showing examples 300 of geometricrules 302, 304 that are supported by the beautification assistant module108 to perform beautification. The examples include endpoint snapping,end tangent alignment, line parallelism, line perpendicularity, linelength equality, arc center snapping, curve offset, curve identify,curve rotation, and curve reflection.

The beautification assistant module 108 treats each of these geometricrules 302, 304 as an extensible set of self-contained geometric rules,each built as a black box and independent of other rules. Each of thegeometric rules 302, 304 represents a single geometric property, such ashaving an endpoint snapped or as being a reflected version of anexisting path.

The input to each rule is an input path 306 including an end-to-endconnected series of Bezier curves, and a set of existing resolved paths308. The beautification assistant module 108, operating as a black box,evaluates the likelihood that the path conforms to the geometricproperty, considering the resolved paths. The beautification assistantmodule 108 then outputs zero or more modified path versions 310, 312.Each modified path version 310, 312 has a corresponding score 314, 316representing a likelihood that the modification is correct. For example,a same-line-length rule would, for input that is a line segment, createoutput versions that are the same lengths as existing line segments,along with scores that indicate how close the segment's initial lengthwas to the modified length. Thus, the scores may reflect an amount ofadjustment performed by respective rules to achieve a respectivesuggestion.

Each geometric rule 302, 304 also has some threshold that determinesthat the score for a modification is too low, and in that case it doesnot output the path. The geometric rules 302, 304 also mark propertiesof the path that have become fixed and therefore can no longer bemodified by future rules. For example, the endpoint-snapping rule marksone or both endpoint coordinates of a path as fixed. Thesame-line-length and parallel-line rules do not attempt to modify asegment with two fixed endpoints. Since the geometric rules 302, 304 donot depend on each other, new geometric rules may be readily added tosupport additional geometric traits.

FIG. 4 depicts an example implementation 400 of successive ruleevaluation and application performed by the beautification assistantmodule 108. Chaining geometric rules can lead to complex modificationsof the input freeform path. Geometric rule application is treated by thebeautification assistant module 108 as branching in a directed rootedtree of paths, where a root node 402 corresponds to the unmodified inputpath. Each branch of the tree corresponds to a unique application of onerule and the branch is given a weight corresponding to the rules score.Therefore, to find a suitable transformations to a user input, thebeautification assistant module 108 traverses down a tree as illustratedin FIG. 4.

In this example, the beautification assistant module 108 evaluates threegeometric rules, endpoint snapping, perpendicularity, and lengthequality. An existing path 404 is fixed and a new path 406 is added, thecombination of which becomes a root node 402 of an evaluation graph andthe expansion begins by testing each of the rules on it. For child node408, for instance, an endpoint snapping geometric rule is applied, forchild node 410 a perpendicularity geometric rule is applied, and forchild node 412 a length equality geometric rule is applied.

A likelihood score is calculated for each rule application, examples ofwhich are illustrated as “1.0,” “0.5,” and “0.75,” respectively. Thetree is expanded using a best-first search scheme, until leaf nodes arereached. An example of this is addition of grandchild nodes 414, 416from child node 408 due to application of perpendicularity and lengthequality geometric rules, respectively. Great-grandchild node 418 isthen formed through application of the perpendicularity rule tograndchild node 416. Thus, each branch of the tree corresponds to aunique ordered application of rules and the branch is given a weightcorresponding to the rules scores of the branch. Therefore, to find asuitable transformations to a user input, the beautification assistantmodule 108 traverses down the tree and bases an adjustment on thecorresponding scores.

Due to significant redundancy in the search space, leaf nodes maycontain duplicate suggestions. Therefore, the beautification assistantmodule 108 is configured to prune the graph during the expansion stepusing information from nodes that were already reached, which isdescribed in further detail in the following.

Formally, assume a node “n^(i)” with Bezier path “p^(i)”, the set ofresolved paths “S,” and the set of all rules “r_(j)ϵR”, an output set“Pi={r_(j)(p^(i),S)}.” A child node “n^(i) _(j)” is created for each“p_(j) ^(i)ϵ{r_(j)(p^(i), S)}”. If “P^(i)” is empty, “n^(i)” is a leafnode. As previously described, each rule outputs a likelihood score forthe transformed path. Since a comparison is performed among differentrules, the likelihood score is normalized into the interval (0,1). Allscores from the nodes visited are used while descending into aparticular leaf node “n” to calculate an overall likelihood score forthe chained transformation, an example of which follows although itshould be readily apparent that other techniques are also usable tocompute the score:

$\begin{matrix}{{\overset{\_}{\mathcal{L}_{i}} = {1 - {\prod\limits_{k = 1}^{d - 1}\left( {1 - {\mathcal{L}\left( {r_{j}\left( {a^{k},S} \right)} \right)}} \right)}}},} & (1)\end{matrix}$where “d” is the depth of “n” in the tree, “a^(k)” is the kth ancestorof “n,” and

(r_(j)(a^(k),S)) denotes the likelihood score from applying rule “r_(j)”to node a^(k).”

The search tree is expanded in a best-first search manner, where theorder of visiting the child nodes is determined by the overall score ofthe node's path from Equation 1 above. While traversing the tree, asuggested set “Q” of leaf nodes are constructed, which is initiallyempty and is filled as the leaf nodes are encountered in the traversal.Once not empty, “Q” helps prune the search. Before a particular subtreeis expanded, the geometric properties of its root is compared with theproperties of each path “qϵQ”.

If each of the tested properties are found in some path “q,” the wholesubtree can be omitted from further processing as shown in an example500 of search graph pruning of FIG. 5. Rules are represented by boxes.An inner node “n” 502 has been expanded into three branches (a,b,c) 504,506, 508. Before further traversal, each of the subtrees stemming fromthe child nodes of “n” are tested against suggestions “qϵQ”. Here,branches “(a)” 504 and “(c)” 506 are fully contained in “q₀” 510 and“q₂” 512, respectively, and thus only branch “(b)” 506 is evaluatedfurther.

In one or more implementations, in order to keep a user from beingexposed to too many suggestions, a size of “Q” is limited. For example,since the graph is traversed in a best-first manner, the search isstopped after finding some number of unique leaf nodes, e.g., ten.

Example Rules and Operations

Geometric transformations are evaluated by the beautification assistantmodule 108 by testing various properties of the new path and a set ofpreviously drawn and processed paths. While testing of some propertiesare relatively simple to perform, others such as path matching involvemore complex processing. The rules are first summarized as described inrelation to FIG. 3, with additional implementation issues includingdetailed description of non-trivial rules are described in thefollowing.

Line Detection

The straightness is estimated by measuring a ratio between length of apath and a distance between its endpoints.

Arc Detection

The approximate curvature is computed by calculating angles betweentangents of successive path sample. A check is performed to determinewhether it is sufficiently uniform and whether the angular span iswithin an expected range, to prevent treating slightly bent lines orspirals as circular arcs. When the span is close to 2 pi or the path isclosed, it is replaced with a full circle.

Endpoint Snapping

A distance between each of the path endpoints and resolved endpoints isexamined. Additionally, snapping to inner parts of the resolved paths isalso examined. Specialized tests based on the properties of linesegments and circular arcs lower the computational complexity of thisoperation.

End Tangent Alignment

If the path endpoint is snapped, the angle between its tangent and thetangent of the point to which it is attached is measured. If thetangents are sufficiently close, the path is adjusted to make theattachment smooth.

Line Parallelism and Perpendicularity

The angle between two line segment paths is compared with the angleneeded to satisfy the parallelism or perpendicularity constraint. Thedistance is taken between the line segments is also taken into accountto slightly increase the priority of nearby paths.

Line Length Equality

The ratio of length of both tested line segments is evaluated. As in theprevious case, the mutual distance is incorporated into a finallikelihood computation.

Arc and Circle Center Snapping

Similar to endpoint snapping, a distance between the current arc centerand potential ones and potential ones are evaluated. In this caseendpoints of other paths, other centers, centers of rotations, andcenters of regular polygons composed from a series of line segments areevaluated.

Path Identity

To detect that two paths have similar shapes, the paths are aligned andtheir discrete Frechet distance used. Different scales are also takeninto account, further discussion of this is included below in the PathMatching section.

Path Offset

Offset paths generalize line parallelism. To detect these paths, thebeautification assistant module 108 progresses along the tested path andmeasures its distance to the reference path, additional discussion ofwhich is included in the Offset Path Detection section below.

Path Rotational Symmetry

For a tested path “x” and resolved reference “y” path of the “sameshape” (as determined by successful application of the path-identityrule), the optimal rotation matrix is found, i.e., center point “c” andangle of rotation “a” that transforms “y” to “y′”. Similarly to arccenter snapping, an attempt is also made to adjust the position of therotation center and snap it to some existing point. Since thesetechniques work in incremental fashion, the angle of rotation isadjusted to the nearest integer quotation of 2 pi, so that additionalpaths can be placed to form full n-fold rotational symmetry.

Path Reflection Symmetry

Similar to rotational symmetry, an axis is found that minimized thedifference between the reflected version of an existing path “y” and thetested path “x.” Nearby existing reflection axes are tested, and anattempt is made the found axis parallel with a coordinate axis.

View-Space Distances

Testing paths for different geometric properties involves measuringlengths and distances. While many path attributes may be compared usingrelative values, absolute values are also necessary, e.g., for snappingendpoints. Using absolute values, however, leads to unexpected behaviorwhen a user interface (e.g., canvas) is zoomed in and out. To eliminatethis problem, distances are computed in view-space pixels, making alldistance tests magnification-independent.

Path Sampling

Analytical work with cubic Bezier curves is challenging. Some practicaltasks, such as finding a path's length or the minimal distance betweentwo paths can only be solved using numerical approaches. Therefore,operations are performed on sampled paths. Since the resolved paths donot change, samples for resolved paths may be precomputed and stored,with sampling limited to new paths. Furthermore, to reduce the memoryand computational complexity of different path comparisons, the samplingmay be simplified, e.g., using a Ramer-Douglas-Peucker algorithm. For apolyline “p”, this finds a reduced version “p′” with fewer points withingiven tolerance “s”, i.e., all points of “p′” lie within the distance“s” of the original path as shown in an example implementation 600 ofFIG. 6. In this example, an original Bezier path 602 is equidistantlysampled, giving a polyline 604. The Ramer-Douglas-Peucker algorithm thenrecursively simplifies the polyline by omitting points closer than “s”to the current approximation as shown at example 606, finallyconstructing a simplified polyline 608. In an example implementation,“ϵ=4” view space pixels at the time the path was drawn is used.

Path Matching

Path matching involves resolving higher-level geometric relations likepath rotational and reflection symmetry. To identify these relations,paths are first classified that are the “same shape,” e.g., paths thatare different instances of the same “template.”

To evaluate the similarity between two sampled paths “p_(a)” and“p_(b),” a similarity measure may be employed, such as a discretevariant of Frechet distances. Formally, it is defined as follows: let(M,d) be a metric space and let the path be defined as a continuousmapping “f:[a,b]→M,” where a, bϵ

, a≤b. Given two paths “f:[a,b]→M,” and “g:[al,bl]→M”, the Frechetdistance “δ_(F)” is defined as:

$\begin{matrix}{{{\delta_{F}\left( {f,g} \right)} = {\inf\limits_{\alpha,\beta}{\max\limits_{t \in {\lbrack{0,1}\rbrack}}{d\left( {{f\left( {\alpha(t)} \right)},{g\left( {\beta(t)} \right)}} \right)}}}},} & (2)\end{matrix}$where “α” (resp. B) is an arbitrary continuous non-decreasing functionfrom [0,1] onto “[a,b] resp. [a′,b′]”.

FIG. 7 depicts an example implementation 700 in which discrete Frechetdistance is shown. The minimum length of the line connecting orderedsets of point samples is shown at 702. Since the resolved paths arestored in the simplified form, the Frechet distance is computed betweenan ordered set of points and an ordered set of line segment rather thanbetween two point sets as shown at 704.

Intuitively, it is usually described using a leash metaphor: a man walksfrom the beginning to the end of one path while his dog on a leash walksfrom the beginning to the end of the other. They can vary their speedsbut they cannot walk backwards. The Frechet distance is the length ofthe shortest leash that can allow them to successfully traverse thepaths.

This can be computed for two point sets using a dynamic programmingapproach. The extension to point and line-segment sets at 704 is thenstraightforward. However, the measure takes into account the absolutepositions of the sample points, while it is the relative distance thatis desired.

FIG. 8 depicts an example implementation 800 showing adjustment of thealignment of two tested paths through first, second, third, and fourthstages 802, 804, 806, 808. First and second paths 810, 812 are receivedat the first stage 802. At the second stage 804, the first and secondpaths 810, 812 are translated and rotated, and universally scaled suchthat endpoints of the curves match.

At the third stage 806, bounding boxes 814, 816 of the respective firstand second paths 810, 812 from the second stage 804 are scaled to matcheach other, which may be perform non-uniformly. The discrete Frechetdistance is computed divided by the length of the tested path to obtainthe relative similarity measure. Because the second path 812 (i.e., the“new” path) might be flipped and/or reversed version of the first path810 (i.e., the “existing” path), four tests are performed between themto determine the correct match.

Thus, for the illustrated example of the first and second paths 810,812, the endpoints of the paths are matched at the second stage 804 andare then scaled so that the bounding boxes match at the third stage 806.If the second “new” path 812 is then evaluated as being sufficientlysimilar to the first “existing” path 810, an adjusted path 818 iscreated as a properly-positioned instance of the first “existing” path810 to replace the second path as shown at the fourth stage 808. This isbut one of a variety of ways to perform path matching.

Offset Path Detection

Offset paths extend the concept of parallelism from line segments topaths. To detect them, a normal line is constructed from each sample ofthe new path. If the line hits an existing reference path, the distanceis measured between the sample point and the closet point on thereference. Note that the distance between the sample point and theline-path intersection is not used, since this would require the user toprecisely draw the approximate offset path.

The measured distance is stored along with its sign, i.e., on which sideof the new path the hit occurred. The hit information is then sortedaccording to the distance, creating a cumulative distribution function,and two values corresponding to (50+/−n)-th percentiles, with “n” beingtwenty-five in an example implementation. By comparing the sign and thedistance values of these samples, the likelihood of the new path beingan offset path of the reference path is calculated.

FIG. 9 depicts an example implementation 900 of offset path detection. Aline is constructed from each point on a sample path 902 in the normaldirection. If an existing reference path 904 is hit, the minimaldistance from the sample to the reference path is calculated as shownusing dashed lines and used in offset-path-likelihood computation.

FIG. 10 depicts an example implementation 1000 showing a transform againfeature. A path is drawn at 1002 and when a new path 1004 is added to auser interface, it is processed and suggestions are output. If the userchooses a suggestion that is a rotation as shown at 1006, a transformagain feature is enabled in which a user can then easily complete theeight-fold rotational symmetry drawing as shown at 1008, e.g., bycontinued drawing and automatic fitting.

Example Results

FIGS. 11, 12, 13, and 14 depicts respective examples 1100, 1200, 1300,1400 of techniques performed by the beautification assistant module 108.Side-by-side pairs show the beautified output on the left and theoriginal input strokes on the right. Thus, as described above thebeautification assistant module 108 supports beautification of freeformdrawings. Since the user input is often imprecise and thus ambiguous,multiple output suggestions may be generated. To this end, the problemis formulated as a search in a rooted tree graph where nodes containtransformed input stroke, edges represent applications of geometricrules and suitable suggestions correspond to different paths from rootnode to some leaf nodes. To avoid computational complexity of traversalthrough the whole graph, a best-first search approach is employed wherethe order of visiting tree nodes is directed by the likelihood ofapplication of the particular geometric rules.

On top of this framework, a system is developed of self-contained rulesrepresenting different geometric transformations, which can be extended.Various rules are implemented that can work not only with simpleprimitives like line segments and circular arcs, but also with generalBezier curves, for which it showed how to detect previously unsupportedrelations such as curve identity or rotational and reflection symmetry.With the ability to process general curves, these techniques extend therange of applicability of freeform sketching, which was conventionallylimited to drawings in specialized, highly-structured applications likeforms or technical diagrams. This advantage is further promoted by thewidespread adoption of touch-centric devices. Further discussion ofthese and other examples is contained in the following section.

Example Procedures

The following discussion describes techniques that may be implementedutilizing the previously described systems and devices. Aspects of eachof the procedures may be implemented in hardware, firmware, or software,or a combination thereof. The procedures are shown as a set of blocksthat specify operations performed by one or more devices and are notnecessarily limited to the orders shown for performing the operations bythe respective blocks. In portions of the following discussion,reference will be made to FIG. 1-.

FIG. 15 depicts a procedure 1500 in an example implementation ofcontrolling beautification of freeform drawings by a computing device.An input is received by a computing device describing a freeform pathdrawn by a user as part of a drawing, the freeform path not formedsolely as a circular arc or a circle (e.g., a fixed distance from apoint) and including one or more curved elements (block 1502). The inputis detectable in a variety of ways by the computing device 102, such asto use touchscreen functionality of the display device 104 (e.g., as agesture made by a finger of a user's hand or stylus), a trackpad, acursor control device (e.g., mouse), and so forth.

The drawing is examined by the computing device to locate another curvedelement in the drawing (block 1504). The beautification assistant module108, for instance, may examine the drawing to locate previously inputfreeform paths or other paths that include curved elements.

One or more suggestions are constructed to adjust the freeform path bythe computing device based on the located curved element in the drawing(block 1506). The suggestions may be based on a curve offset, curvesimilarity, and so on as described above. The constructed one or moresuggestions are output to adjust the freeform path by the computingdevice (block 1508). The user may be provided with the suggestions inthe user interface that are selectable, the suggestions may be used toautomatically adjust the freeform path, and so forth.

FIG. 16 depicts a procedure 1600 in another example implementation ofcontrolling beautification of freeform drawings by a computing device.An input is received by a computing device describing the freeform pathdrawn by a user as part of a drawing (block 1602). The drawing is alsoexamined by the computing device to locate other paths that alreadyexist in the drawing (block 1604). A user, for instance, may draw afreeform path 204 (e.g., using a finger, stylus, mouse, and so on) in adrawing that already includes an existing freeform path 202. Thecomputing device may thus output a display of the freeform path andlocate other paths that are already drawn, e.g., by the user orotherwise already exist in drawing.

A likelihood is evaluated that the freeform path conforms individuallyto respective ones of a plurality of geometric rules, one or more ofwhich consider the other paths that already exist in the drawing suchthat the evaluation performed by at least one of the geometric rulescauses output of a modification of the freeform path and a scorerepresenting a likelihood that the modification is correct (block 1606)The beautification assistant module 108, for instance, may form adirected graph as shown in FIG. 4 by treating each of a plurality ofgeometric rules 302, 304 as an extensible set of self-containedgeometric rules, each built as a black box and independent of otherrules that outputs a score representing the likelihood that themodification is correct. The geometric rules may also be chained tosupport complex modifications of the input freeform path throughsuccessive application of the geometric rules to arrive at modificationsand scores for those modifications.

One or more suggestions are constructed to adjust the freeform path bythe computing device based on the scores generated by the rules (block1608). The one or more suggestions, for instance, may be based on themodifications and corresponding score representing the likelihood thatthe modification is correct. The constructed suggestions are then outputto adjust the freeform path by the computing device (block 1610) and assuch may act to beautify the paths in the drawing input by a user basedon other paths that already exist in the drawing. A variety of otherexamples are also contemplated as described above.

Example System and Device

FIG. 17 illustrates an example system generally at 1700 that includes anexample computing device 1702 that is representative of one or morecomputing systems and/or devices that may implement the varioustechniques described herein. This is illustrated through inclusion ofthe freeform drawing module 106. The computing device 1702 may be, forexample, a server of a service provider, a device associated with aclient (e.g., a client device), an on-chip system, and/or any othersuitable computing device or computing system.

The example computing device 1702 as illustrated includes a processingsystem 1704, one or more computer-readable media 1706, and one or moreI/O interface 1708 that are communicatively coupled, one to another.Although not shown, the computing device 1702 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 1704 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 1704 is illustrated as including hardware element 1710 that maybe configured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 1710 are not limited by the materials from whichthey are formed or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable storage media 1706 is illustrated as includingmemory/storage 1712. The memory/storage 1712 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage component 1712 may include volatile media (such as randomaccess memory (RAM)) and/or nonvolatile media (such as read only memory(ROM), Flash memory, optical disks, magnetic disks, and so forth). Thememory/storage component 1712 may include fixed media (e.g., RAM, ROM, afixed hard drive, and so on) as well as removable media (e.g., Flashmemory, a removable hard drive, an optical disc, and so forth). Thecomputer-readable media 1706 may be configured in a variety of otherways as further described below.

Input/output interface(s) 1708 are representative of functionality toallow a user to enter commands and information to computing device 1702,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone, a scanner, touch functionality (e.g., capacitiveor other sensors that are configured to detect physical touch), a camera(e.g., which may employ visible or non-visible wavelengths such asinfrared frequencies to recognize movement as gestures that do notinvolve touch), and so forth. Examples of output devices include adisplay device (e.g., a monitor or projector), speakers, a printer, anetwork card, tactile-response device, and so forth. Thus, the computingdevice 1702 may be configured in a variety of ways as further describedbelow to support user interaction.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 1702. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and non-volatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing mediumthat is configured to transmit instructions to the hardware of thecomputing device 1702, such as via a network. Signal media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Signal media also include anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 1710 and computer-readablemedia 1706 are representative of modules, programmable device logicand/or fixed device logic implemented in a hardware form that may beemployed in some embodiments to implement at least some aspects of thetechniques described herein, such as to perform one or moreinstructions. Hardware may include components of an integrated circuitor on-chip system, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a complex programmable logicdevice (CPLD), and other implementations in silicon or other hardware.In this context, hardware may operate as a processing device thatperforms program tasks defined by instructions and/or logic embodied bythe hardware as well as a hardware utilized to store instructions forexecution, e.g., the computer-readable storage media describedpreviously.

Combinations of the foregoing may also be employed to implement varioustechniques described herein. Accordingly, software, hardware, orexecutable modules may be implemented as one or more instructions and/orlogic embodied on some form of computer-readable storage media and/or byone or more hardware elements 1710. The computing device 1702 may beconfigured to implement particular instructions and/or functionscorresponding to the software and/or hardware modules. Accordingly,implementation of a module that is executable by the computing device1702 as software may be achieved at least partially in hardware, e.g.,through use of computer-readable storage media and/or hardware elements1710 of the processing system 1704. The instructions and/or functionsmay be executable/operable by one or more articles of manufacture (forexample, one or more computing devices 1702 and/or processing systems1704) to implement techniques, modules, and examples described herein.

The techniques described herein may be supported by variousconfigurations of the computing device 1702 and are not limited to thespecific examples of the techniques described herein. This functionalitymay also be implemented all or in part through use of a distributedsystem, such as over a “cloud” 1714 via a platform 1716 as describedbelow.

The cloud 1714 includes and/or is representative of a platform 1716 forresources 1718. The platform 1716 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 1714. Theresources 1718 may include applications and/or data that can be utilizedwhile computer processing is executed on servers that are remote fromthe computing device 1702. Resources 1718 can also include servicesprovided over the Internet and/or through a subscriber network, such asa cellular or Wi-Fi network.

The platform 1716 may abstract resources and functions to connect thecomputing device 1702 with other computing devices. The platform 1716may also serve to abstract scaling of resources to provide acorresponding level of scale to encountered demand for the resources1718 that are implemented via the platform 1716. Accordingly, in aninterconnected device embodiment, implementation of functionalitydescribed herein may be distributed throughout the system 1700. Forexample, the functionality may be implemented in part on the computingdevice 1702 as well as via the platform 1716 that abstracts thefunctionality of the cloud 1714.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. A method of controlling beautification of afreeform path by at least one computing device, the method comprising:receiving, by the at least one computing device, an input describing thefreeform path drawn in response to a user input as part of a drawing,the freeform path not formed solely as a circular arc or a circle andincluding at least one curved element; locating, by the at least onecomputing device, a plurality of existing paths included in the drawingwith the freeform path, the freeform path having at least one geometricproperty similar to the plurality of existing paths in the drawing;evaluating, by the at least one computing device, a similarity betweenthe freeform path and the plurality of existing paths in the drawing,the evaluating based, in part, on a geometric rule that corresponds tothe at least one geometric property; constructing, by the at least onecomputing device, at least one suggestion based on the geometric ruleand aging of an existing path of the plurality of existing pathsrelative to aging of one or more other existing paths of the pluralityof existing paths, the constructing including; scaling either thefreeform path or the existing path, the scaling causing a distancebetween endpoints of the freeform path to match a distance betweenendpoints of the existing path; or causing a curvature of a segment ofthe freeform path to match a curvature of a segment of the existingpath; and outputting, by the at least one computing device, theconstructed at least one suggestion to modify the freeform path.
 2. Amethod as described in claim 1, further comprising constructing anothersuggestion that causes the freeform path to assume a curve offset inrelation to the existing path in the drawing.
 3. A method as describedin claim 2, wherein the constructing of the other suggestion furtherincludes measuring a distance from the freeform path to the existingpath in the drawing and locating portions where the distance isrelatively uniform and the other suggestion describes the curve offset.4. A method as described in claim 1, further comprising constructinganother suggestion as a copy of the existing path in the drawing thathas been translated, scaled, or rotated.
 5. A method as described inclaim 1, further comprising constructing another suggestion based oncurve similarity of the at least one curved element of the freeform pathto the existing path in the drawing.
 6. A method as described in claim5, wherein the curve similarity is determined by scaling and aligningendpoints of the freeform path to the existing path in the drawing,measuring a distance metric between the scaled and align freeform pathto the existing path in the drawing, scaling an absolute value of thedistance metric by a length of the freeform path, and replacing thefreeform path with the existing path in the drawing in response to adetermination that the absolute value is within a defined threshold. 7.A method as described in claim 6, wherein the distance metric is aFrechet distance.
 8. A method as described in claim 1, wherein screenspace in pixels is used for distances in the constructing of the atleast one suggestion.
 9. A method as described in claim 1, wherein theat least one suggestion is based at least in part on distances betweenthe freeform path and respective ones of the plurality of existingpaths.
 10. A method as described in claim 1, wherein the method furthercomprises: calculating one or more scores representing a likelihood thatthe at least one suggestion is correct; and outputting the one or morecalculated scores.
 11. A method as described in claim 10, wherein theone or more scores reflect an amount of adjustment performed to generatethe constructed at least one suggestion to modify the freeform path. 12.A method of controlling beautification of a freeform path by at leastone computing device, the method comprising: receiving, by the at leastone computing device, an input describing the freeform path drawn by auser as part of a drawing, the freeform path including at least onecurved element; evaluating, by the at least one computing device, alikelihood that the freeform path conforms individually to respectiveones of a plurality of geometric rules, the evaluating including:identifying tangent lines of the curved element at multiple locationsalong a segment of the freeform path; calculating angles betweensuccessive tangent lines of the identified tangent lines; anddetermining whether the calculated angles between the successive tangentlines of the freeform path is within an expected range; constructing, bythe at least one computing device, at least one suggestion to adjust thefreeform path by the computing device based, in part, on thedetermination that the calculated angles between the successive tangentlines of the freeform path are within the expected range, the at leastone suggestion having a uniform curvature throughout the segment of thefreeform path; and outputting, by the at least one computing device, theat least one constructed suggestion to adjust the freeform path suchthat the curved element is a full circle.
 13. A method as described inclaim 12, wherein the geometric rules include a threshold to determinewhether to output the at least one suggestion to adjust the freeformpath.
 14. A method as described in claim 12, wherein the plurality ofgeometric rules are configured to mark properties of the freeform paththat have become fixed due to modification and are no longer modifiableby other ones of the plurality of geometric rules.
 15. A method asdescribed in claim 12, wherein the evaluating including chaining of atleast a portion of the plurality of geometric rules through branching ina directed rooted tree of paths.
 16. A method as described in claim 12,wherein the method further comprises: calculating one or more scoresrepresenting a likelihood that the at least one suggestion is correct;and outputting the one or more calculated scores.
 17. A method asdescribed in claim 16, wherein the one or more scores reflect an amountof adjustment performed to generate the constructed at least onesuggestion to adjust the freeform path.
 18. A computing devicecomprising: at least one module implemented at least partially inhardware including a processing system and computer-readable storagemedia, the at least one module configured to perform operationscomprising: receiving an input by the computing device describing afreeform path drawn in response to a user input as part of a drawing,the freeform path including at least one curved element; locating, bythe computing device, an existing path in the drawing that includes thefreeform path, the freeform path having at least one geometric propertysimilar to the existing path in the drawing; evaluating, by thecomputing device, a path rotational symmetry between the freeform pathand the existing path in the drawing, the evaluating including;identifying a common center point of a rotational center between thefreeform path and the existing path; and determining an angle ofrotation that minimizes differences between the freeform path and theexisting path; and constructing, by the computing device, at least onesuggestion to adjust the freeform path based on the evaluating causingthe freeform path to rotate around the common center point.
 19. Acomputing device as described in claim 18, further comprisingconstructing another suggestion by measuring a distance from thefreeform path to the existing path in the drawing and locating portionswhere the distance is relatively uniform and in such instances the othersuggestion is configured to adjust the curve of the freeform path or theexisting path causing the freeform path and the existing path to have asimilar shape .
 20. A computing device as described in claim 18, whereinthe common center point is snapped to an existing point of the drawing.